EndoSensorFusion: Particle Filtering-Based Multi-sensory Data Fusion with Switching State-Space Model for Endoscopic Capsule Robots using Recurrent Neural Network Kinematics

A reliable, real time multi-sensor fusion functionality is crucial for localization of actively controlled nextgeneration endoscopic capsule robots, as an emerging minimally invasive diagnostic technology for the inspection of gastrointestinal (GI) tract and diagnosis of a wide range of diseases and pathologies. In this study, we propose a novel multi-sensor fusion approach based on switching observations model using non-linear kinematics learned by recurrent neural networks for real-time endoscopic capsule robot localization. Our method concerns the sequential estimation of a hidden state vector from noisy pose observations delivered by multiple sensors, a 5 degree-of-freedom (5-DoF) absolute pose estimation by magnetic 2D Hall-effect sensor array and a 6-DoF relative pose estimation by a visual odometry approach. In addition, the proposed method is capable of detecting and handling sensor failures in-between nominal sensor states. Detailed analyses and evaluations made using ex-vivo experiments on a porcine stomach model prove that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.